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LARGE  COMPLEX  DATA  SETS 


A major  problem  in  underwater  acoustic  research  is  j 

the  search  for  better  understanding  of  the  effects  of 
large  scale  oceanographic  processes  that  either  enhance 
or  destroy  coherence.  The  search  for  these  effects 
requires  the  digestion  of  large  complex  data  sets.  The 
major  efforts  under  this  contract  were  in  two  areas  of 

research  aimed  at  enhancing  our  ability  to  extract  in-  | 

formation  from  such  data.  j 

) 

1.1  Complex  Information  Display  Using  Color 

This  long  term  task  was  aimed  at  determining  the  j 

spectral  phase  stability  of  noise  and  man-made  signals.  ! 

The  color  display  facility  is  a 256  x 256  color  point 

display  under  computer  control.  Although  quite  general  j 

in  nature,  its  only  use  has  been  to  study  the  spectrum  ; 

as  defined  by  256  adjacent  bins  of  an  FFT  analysis, 
displaying  each  analysis  below  the  previous.  The  spectral 

magnitude  is  the  displayed  brightness,  and  the  spectral  ! 

phase  is  the  color.  1 

The  hardware  (electronics  and  mechanical)  had  been 
a major  problem  since  the  facilities  conception,  and  it 
was  not  until  early  1976  that  the  facility  was  deemed 
reliable,  and  the  display  free  of  artifacts.  (For  example, 

memory-timing  problems  caused  vertical  black  lines.)  i 
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A modest  operating  system  was  written,  tested, 
and  completed  for  control  of  the  display.  An  effective 
phase-to-color  coding  was  developed  by  trial  and  re- 
finement; this  code  dontains  no  "white"  as  only  one  or 
two  primary  colors  are  excited  for  each  display  point. 

This  code  appears  to  be  as  effective  in  conveying  phase 
and  amplitude  information  as  I had  hoped  it  would  be. 

The  data  used  for  development  were  taken  in  January 
of  1974  in  the  MIMI  Atlantic  test.  The  most  obvious 
conclusion  was  the  need  for  time  overlap  (each  FFT  con- 
taining 25  percent  new  data  and  75  percent  old  data) . 

This  agreed  with  the  black  and  white  "gram"  display  con- 
clusion. Several  of  the  test  records  showed  remarkable 
signal  phase  stability;  however,  it  was  by  chance  that 
the  signals'  frequency  coincided  (nearly)  with  the  center 
of  an  FFT  bin,  or  this  would  not  have  been  observed. 

Signal  processing  programs  to  correct  this,  allowing 
operator  directed  cueing  and  controlled  f ine-grain-tuning , 
were  written  at  the  close  of  the  contract.  This  work  is 
expected  to  continue  under  other  ONR  contracts. 

1.2  A Data  Processing  Language 

Work  was  begun  under  Dr.  Gerald  Cederquist  on  a 
general  data  processing  language  that  would  allow  easier 
manipulation  of  large  data  bases,  such  as  the  January  1974 
MIMI  test,  with  greater  assurance  of  accuracy.  This 
development  was  discontinued  when  Dr.  Cederquist  neared 


r 


! 

I 

the  end  of  his  post-doctoral  stay.  Unhappily,  the  concepts 
that  were  beginning  to  gel  were  not  sufficiently  well  j 

formulated  to  warrant  reporting;  we  are,  however,  a bit 
smarter  than  we  were.  We  understand  how  the  use  of 
more  complete  descriptors  aid  in  analysis  of  data,  and 
some  of  the  complexities  that  must  be  overcome  in  des- 
criptor transformations  as  data  are  processed. 
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2.  SIGNAL  PROCESSING 


Extension  of  the  theoretical  basis  for  signal 
processing,  and  conversion  of  theory  to  practice  have 
been  low  level,  but  continuing  efforts,  throughout  this 
contract.  Two  such  efforts  warrant  reporting  here. 

2.1  Factor  Inverse  Filtering 

Under  this  contract  and  another  ONR  contract  the 
basis  for  our  previous  success  using  pseudorandom  linear 
maximal  binary  codes  for  propagation  has  been  examined, 
and  the  techniques  generalized.  This  has  been  a process 
that  evolved  over  many  years,  and  we  developed  and  used 
the  techniques  before  we  understood  them.  Early  in  1975 
I began  to  crystallize  this  into  "factor  inverse  filtering" 
and  have  tested  it  in  three  experiments.  There  will  be 
both  a report  on  this  (under  the  other  ONR  contract)  and 
papers,  but  the  theory  is  essentially  this: 

Suppose  a propagation  experiment  is  to  be  conducted 
from  one  transmitter  to  one  or  more  receivers,  with  the 
objective  of  measuring  the  channel  transfer  function 
C(f,r)  from  the  transmitter  to  the  rth  receiver.  In 
principle  we  would  like  to  transmit  a short  pulse,  with 
spectrum  P(f)  , using  due  care  to  match  to  the  trans- 
ducer's transfer  function  X(f).  If  we  could  measure 
the  channel  pulse  response,  P(f)C(f,r)  , we  would  be 
happy,  but  the  transducer  cannot  deliver  sufficient 
energy  in  one  short  pulse  to  overcome  the  noise  and 
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forward  scattered  reverberation.  Therefore  we  use  a 
coded  signal  with  spectrum  S(f)  instead. 

The  key  fact  is  that  we  have  always  used,  and  should 
use,  a signal  that  contains  the  hypothetical  pulse 
spectrum  as  a factor,  and  the  actual  signal  has  spectral 
zeros  only  where  that  pulse  spectrum  would  be  zero.  For- 
mally 

S(f)  = P(f)G(f)  , G(F)  ^ 0 anywhere 
The  actual  received  spectrum  at  the  rth  receiver  is 

R(f,r)  = [p(f)G(f)  X(f)  C(f,r)  +^N(f ,r)]  A(f,r) 

where  A(f,r)  is  the  effect  of  the  amplifiers  and  filters 
for  the  rth  hydrophone,  and  the  square  root  on  the  noise 
power  spectrum,  N(f)  , indicates  the  Fourier  transform 
of  the  actual  noise  on  the  rth  receiver  during  the  measure- 
ment. If  there  were  no  noise,  we  would  simply  solve  for 
the  desired  result,  P(f)C(f,r)  . Do  so. 

Estimated  [p(f)C(f,r)]  = R(f,r)  / G ( f ) X ( f ) A ( f , r) 

If  a laboratory  or  "close  in"  test  permits,  measure  the 
system  without  the  ocean. 

Test:  T(f,r)  = S(f)  X(f)  A(f,r) 


The  denominator  in  the  estimation  is  P(f)/T(f,r)  , which 
will  have  no  zeros.  If  the  signal  is  chosen  carefully. 
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G(f)  will  be  fairly  constant  in  magnitude,  and  so  will 
this  denominator.  The  phase  function  of  this  denominator 
is  most  important,  as  it  undoes  the  "smearing"  done  by 
the  transducer  and  the  receiving  filters.  More  important, 
if  array  work  is  being  done,  the  phases  of  the  individual 
hydrophones  will  be  corrected  for  individual  receiver 
channel  differences.  It's  really  simpler  to  use  than  it 
is  to  prove.  The  theory  accounts  for  the  signal-to-noise 
ratio  gain  due  to  increased  transmission  time  and  loss 
due  to  not  matched-filtering,  all  in  terms  of  the  "flatness 
of  the  factor  spectrum,  G(f)  . 

Power  Spectrum  Measurements 
We  have  been  wrestling  with  the  theories  and  practice 
of  power  spectrum  measurements  for  many  years.  Either 
our  theory  is  inadequate  or  our  practice  is  archaic, 
probably  both.  Under  the  previous  ONR  contract  Dr.  J.  0. 
Gobien  developed  the  theoretically  optimum  measurement 
method  for  wide  sense  stationary  Gaussian  processes  with 
"poles-only"  rational  power  spectra  using  continuous 
measurements  (not  sampled) , and  optimum  in  the  Baysian 
sense.  That  work  had  some  drawbacks:  it  required  perfect 

analog  differentiators,  and  the  stochastic  theory  claimed 
the  high  frequency  falDoff  was  "singu3ar]y  estimatable , " 
i.e.,  should  be  immediately  obvious. 
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Under  the  present  contract  a doctoral  study  by 
Ron  Carpinella  analyses  the  practical  implications  of 
Gobien's  work  when  applied  to  high-sampling-rate  digital 
analysis,  compared  to  Berg's  "maximum  entrophy  method" 
which  applies  to  low-sampling-rate  digital  analysis. 

Berg's  technique  may  be  treated  as  assuming  an  auto- 
regressive process,  which  is  very  close  in  nature  to 
that  treated  by  Gobien. 

This  has  been  a difficult  study.  The  mathematical 
models  are  elegant,  obtuse,  and  difficult  to  manipulate; 
yet  they  appear  to  be  inadequate  for  studying  measurable 
features  of  underwater  acoustic  noise.  The  FFT  and 
third-octave  band  analyses  may  be  good  practical  tech- 
niques, but  they  have  no  support  in  stochastic  theory. 

At  the  close  of  this  contract  Carpinella  was  beginning 
to  sort  the  details  from  the  principles.  This  work  will 
continue  unsupported,  and  any  practical  results  will  cer- 
tainly show  up  in  subsequent  work  under  other  ONR  contracts. 
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Appendix  A 


TECHNICAL  REPORTS  AND  MEMORANDUMS 


The  following  technical  report  (thesis)  and  memo- 
randums were  produced  wholly  or  in  part  under  Contract 
N00014-75-C-0174 . 


1.  G.  N.  Cederquist,  The  Use  of  Computer-Generated 
Pictures  to  Extract  Information  from  Underwater 
Acoustic  Transfer  Function  Data,  Cooley  Electronics 
Laboratory  Technical  Report  No.  227,  The  University 
of  Michigan,  Ann  Arbor,  April  1975.  ADAOlO  229. 

2.  G.  N.  Cederquist,  CONSYS ; A Collection  of  FORTRAN 
Subroutines  to  Produce  Contour  Maps  of  Data  Surfaces 
Defined  on  Rectangular  Grids,  Cooley  Electronics 
Laboratory  Technical  Memorandum  No.  112,  July  1976. 
ADA027  871 

3.  G.  N.  Cederquist,  PERSYS;  A Collection  of  FORTRAN 
Subroutines  to  Produce  Perspective  Views  of  Data 
Surfaces  Defined  on  Rectangular  Grids,  Cooley 
Electronics  Laboratory  Technical  Memorandum  No. 

113,  August  1976.  ADA030  152 
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Appendix  B 


-ONFERENCE  PAPERS  AND  SYMPOSIA 


The  following  conference  papers  and  symposium  pre- 
sentations were  supported  wholly  or  in  part  Dy  Contract 
N00014-75-C-0174. 


1.  G.  N.  Cederquist,  "The  FLECS  Preprocessor  for 
FORTRAN,"  Computing  Center,  The  University  of 
Michigan,  Ann  Arbor,  October  6,  197b. 

2.  G.  N.  Cederquist,  "Program  Microstructuring," 
Computing  Center,  The  University  of  Michigai,  Ann 
Arbor,  January  15,  1976. 

3.  G.  N.  Cederquist,  "A  Survey  of  Recent  Practical 
Developments  in  the  Art  of  Program  Constructic.i , " 
Western  Michigan  University,  Kalamazoo,  South- 
western Chapter  of  Association  for  Computing 
Machinery,  February  18,  1976. 

4.  G.  N.  Cederquist,  "An  Interactive  System  for 
Time  Series  Analysis  and  Display  of  Water  Quality 
Data,"  5th  International  CODATA  Conference, 
International  Council  of  Scientific  Unions, 
Denver,  Colorado,  June  28,  1976. 
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Rosenstiel  School  of  Marine  and 
Atmospheric  Sciences 
4600  Rickenbacker  Causeway 
Miami,  Florida  33149 

Westinghouse  Electric  Corporation 
Advanced  Development  Programs 
Marketing  Department  - MS  227 
P.O.  Box  746 

Baltimore,  Maryland  21203 
ATTN:  F.  J.  Frissyn 
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Oak  Ridge  National  Laboratory 
Union  Carbide  Corporation 
Nuclear  Division 
P.O.  Box  X 

Oak  Ridge,  Tennessee  37830 

Dr.  C.  V.  Kimball 
University  of  Miami 
Rosenstiel  School  of  Marine  and 
Atmospheric  Sciences 
Miami,  Florida  33149 

Cooley  Electronics  Laboratory 
University  of  Michigan 
Ann  Arbor,  Michigan  48109 


